One task faced in the field of computer vision is tracking the movement of an object within a video scene. In many cases, tracking the object consists of tracking the location and orientation of the object with respect to a camera—that is, the point in space that is the perspective from which the video scene is captured.
One significant subset of the movement tracking task is tracking the movement of a roughly planar, i.e., 2-dimensional, surface within a video scene. Such a surface may be one side of a sheet of paper, either a free-standing sheet of paper or a sheet of paper in a book. It may also be a surface presented by virtually any other object. One application of surface tracking is introducing a view of a virtual, 3-dimension object into each frame of the video scene that appears to float about the surface, termed reality augmentation.
Surface tracking has conventionally been addressed through the use of fiducial marker tracking. Fiducial marker tracking involves affixing a special, visually distinct symbol (such as a bold black square having a white interior) to the surface to be tracked; locating the symbol—called a fiducial marker—in each video frame; determining the location of the surface with respect to the camera based on the size of the marker in the video frame and its location in the video frame; and determining the orientation of the surface with respect to the camera based on the orientation of the marker in the video frame.
Fiducial marker tracking has significant disadvantages, however. First, fiducial marker tracking fails any time the fiducial marker is not present in the scene. This can happen in a variety of circumstances, even while the surface remains in the scene. For example, the fiducial marker may be occluded by an object that intervenes in space between the camera and the fiducial marker, such as a person's hand or another inanimate object. The fiducial marker may also move outside of the boundaries of the video scene, such as when the camera moves so close to (or zooms in so close to) the surface that the scene can only contain a portion of the surface that excludes the fiducial marker. This disadvantage can only be overcome by adding larger and larger numbers of fiducial markers to the surface, overwhelming the appearance of the surface and obscuring its innate visual features.
Second, fiducial marker tracking can fail when the surface is distant from the camera, and the camera's view of the surface is too limited to distinguish the fiducial marker. The only way to address this disadvantage is to use larger and larger fiducial markers, again overwhelming the appearance of the surface and obscuring its innate visual features.
In view of these shortcomings, an approach to surface tracking that does not rely on the continuous visibility and distinguishability of fiducial markers affixed to the surface would have significant utility.